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Abstract 


The ALS Metric is the predominant tool for predicting the cost of ALS 
systems. Metric goals for the ALS Program are daunting, requiring a 
threefold increase in the ALS Metric by 2010. Confounding the problem, the 
rate new ALS technologies reach the maturity required for consideration in 
the ALS Metric and the rate at which new configurations are developed is 
slow, limiting the search space and potentially giving the perspective of a 
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ALS technology, the ALS Metric may remain elusive. 


This paper is a sequel to a paper published in the proceedings of the 2003 
ICES conference entitled, "Managing to the metric: an approach to optimizing 
life support costs." The conclusions of that paper state that the largest 
contributors to the ALS Metric should be targeted by ALS researchers and 
management for maximum metric reductions. Certainly, these areas potentially 
offer large potential benefits to future ALS missions; however, the ALS 
Metric is not the only decision-making tool available to the community. 

To facilitate decision-making within the ALS community a combination of 
metrics should be utilized, such as the ESM-based ALS metric, but also those 
available through techniques such as life cycle costing and faithful 
consideration of the sensitivity of the assumed models and data. Often a 
lack of data is cited as the reason why these techniques are not considered 
for utilization. An existing database development effort within the ALS 
community, known as OPIS, may provide the opportunity to collect the 
necessary information to enable the proposed systems analyses . A review of 
these additional analysis techniques is provided, focusing on the data 
necessary to enable these. The discussion is concluded by proposing how the 
data may be utilized by analysts in the future. 


Introduction 


In his paper entitled, "Managing to the metric: an approach to optimizing 
life support costs, " Drysdale astutely identifies that several aspects of the 
current perspective on Advance Life Support (ALS) system design are 
significantly costlier than some other aspects of the ALS system (Drysdale, 
2003). Some of the costlier aspects of the system include the cabin volume, 
the prepackaged food, expendable clothing, and the preparation of food, among 
several others. However, Drysdale notes that ALS priorities apparently lie 
elsewhere. Current ALS Metric improvements are not in areas where the ALS 
Metric costs were highest and in some cases are achieved in areas where 
minimal impact on the overall ALS Metric will be seen, no matter how large 
the improvements may be. Drysdale then notes that if large gains are 
necessary to achieve the ALS Metric goals for the ALS Program, then the large 
contributors must be targeted. If the ALS Metric goals cannot be achieved in 
this manner, then it is not likely that they can be achieved at all. In 
conclusion, Drysdale proposes several changes in ALS System design, focusing 
on the large contributors to the ALS Metric and shows that it may be possible 
to achieve the ALS Metric goals - a positive sign for the ALS Program. 

Evidently, ALS priorities are not solely focused on achieving metric goals. 

To ascertain what the priorities of the ALS Program are it is necessary to 



consider the overall goals of the project, how these goals contribute to the 
goals of other NASA entities, the goals greater of NASA, and national 
priorities. From the topmost level, a renewed exploration spirit has been 
initiated nationwide by the President (Bush, 2004) . This has initiated a 
reorganization of NASA. From this statement by the president ensued a 
particularly relevant creation of a new office 1 , "The Office of Exploration 
Systems." The responsibilities of this new office shall be to set priorities 
and direct the identification, development, and validation of exploration 
systems and related technologies (NASA, 2004a) . This falls directly into the 
NASA goals set by the NASA 2003 Strategic Plan, "Extend the duration and 
boundaries of human space flight to create new opportunities for exploration 
and discovery," (NASA, 2003). The President's vision directs that an 
expedition to the lunar surface occur between 2015 and 2020, where testing of 
new approaches, technologies, and systems shall occur in an effort to support 
sustained human exploration to Mars and beyond (Bush, 2004; NASA, 2003) . 

The ALS Program is a small but critical portion of NASA working towards 
achieving the above goals. Within NASA, the ALS Program is charged with the 
research and development of the technologies to enable water purification, 
air revitalization, processing of solid waste materials, as well as providing 
a food source for the crew (NASA, 2004b) . The ALS Project Plan lists five 
project objectives (Russo and Henninger, 2002) : 

1. Provide Advanced Life Support technologies that significantly 
reduce life cycle costs, improve operational performance, promote 
self-sufficiency, and minimize expenditure of resources for long- 
duration missions. 

2. Develop and apply methods of systems analysis and engineering 
to guide investments in technology, resolve and integrate 
competing needs, and guide evolution of technologies. 

3 . Resolve issues of microgravity performance through space 
flight research and evaluation. 

4. Ensure timely transfer of new life support technologies to 
missions . 

5. Transfer technologies to industrial and residential sectors 
for national benefit. 

ALS objectives are well aligned with the directives handed down by the 
President and the NASA Strategic Plan. To enable Martian missions, the ALS 
Program needs to produce technologies than enable such missions that are low 
in cost, highly efficient, self sufficient, and minimize resource 
utilization 2 . In particular, life cycle costs (LCC) are specified by the ALS 
Program objectives. The ALS Metric, a figure of merit acting as a proxy for 
launch costs, is certainly a significant portion of the life cycle costs of 
the system, however it does seem that a significant portion of LCC are 
currently not considered. Often it is asserted that the necessary data 
required for life cycle costing is unavailable. In another paper dealing 
with decision support for ALS, Jones presents existing models currently 


1 Other changes that were proposed by the President's directive include: the 
retirement of the Shuttle, development of a new manned exploration vehicle, a 
plan to return to the Moon by 2015 with both robotic and human missions, and 
an increased use of robotic missions for space exploration (Bush, 2004) . 

2 Perhaps ALS is not as well aligned with respect to the precursor Lunar 
missions since most technology development and analysis has considered 
Martian missions as the target, however Lunar missions may be utilized as 
testbeds for Martian technology. This remains to be seen. 



utilized by NASA and the Air Force to estimate costs such as Design, 
Development, Testing, and Evaluation (DDT&E) Costs; Launch and Emplacement 
Costs; and Operations Costs (Jones, 2003) . The models are top-level in 
nature, likely due to the limited available data to make such predictions. 
Nonetheless, Jones was able to show that according to the best models 
currently available launch costs are not necessarily the largest portion of 
the LCC of ALS missions; in fact, based on these models research and 
development costs easily rival the costs of launching ALS systems for high 
complexity items such as avionics. However, these models may not have been 
developed considering ALS missions to the Moon or Mars. Thus, it may be less 
certain that for life support logistics items like bulk gases, water, and 
food are so costly to develop. Furthermore, launch costs will reach 
unprecedented highs for missions to Mars due to the extreme differences. 

A more comprehensive analysis would be desirable for the ALS (and the greater 
NASA) community. Although, LCC is an attractive option for industry, it 
seems that some aspects are not currently tractable to a level high enough to 
merit the additional expenditure 3 . However, the current approach of systems 
engineering and analysis based almost entirely on the equivalent system mass 
(ESM) based ALS Metric is also lacking, especially considering that 
management decision making is evidently not solely based on metric 
improvements. Significant portions of the costs of ALS missions are not 
being considered in analysis and decision-making may benefit from the results 
from such analysis. To address this need, some techniques often utilized or 
related to LCC can be added relatively easily to the repertoire of ALS 
systems analysts. Some of these techniques are briefly outlined here with 
their data requirements. In addition, since data seems to be lacking, a plan 
is set forth to collect the necessary data for future analysis within ALS and 
NASA with some considerations for use of the data, once it is collected. 

Analysis Techniques and Data Requirements 

LCC is a very comprehensive analysis technique attempting to consider the 
costs of all aspects of a system from the time the analysis of a new idea 
begins, through the building and operations phase, and ultimately until the 
project is disposed of. The determination of metrics, such as the ALS 
Metric, is a critical intermediary step in the LCC process used for decision- 
making. To improve decision making in ALS, it may be enough to add some 
metrics to the analyst repertoire. Ideally, we would focus on metrics that 
deal with aspects other than launch costs, such as those highlighted by 
Jones. Models such as those utilized by Jones should be adequate, until 
there is better data. Two short-term improvements to these models would be 
useful. (1) Consider the original design uses of these models. If they are 
not suitable for ALS missions to the Moon or Mars, they should be upgraded in 
some simplistic manner or all results reported should highlight the potential 
shortcomings. (2) Attempt to quantify the uncertainty in their predictions 
based on the uncertainty in input data. Rodriguez et al . (2003a) provide 

techniques for considering the uncertainty in model output. 

Developing a cost breakdown structure (CBS) that specifically identifies all 
costs to be incurred throughout the system life cycle would be a good way to 
increase the breadth of ALS systems studies and potentially providing a 
complete LCC analysis in the future (Blanchard and Fabrycky, 1998) . This 
requires that models are be developed to predict the each cost incurred 


3 For a comprehensive description of LCC, refer to Blanchard and Fabrycky 
(1998). 



throughout the life cycle. It is precisely the lack of these models that is 
preventing the preparation of a comprehensive LCC in ALS. With such models, 
the models described by Jones could be replaced or upgraded. ESM, for 
example, could be utilized as a basis in development of a model to predict 
the costs of launch. Costs of DDT&E, particularly for software, are harder 
to ascertain 4 . Some considerations for double booking of costs will be 
required. For example, ESM currently considers many aspects of operations 
through crew time, logistics, and power. For a complete LCC, and perhaps in 
preliminary stages, it may be beneficial to have these aspects parsed out for 
greater resolution in the analysis. 

Research, Development, Test and Evaluation 5 life cycles are outlined in the 
literature by Patterson (1999) . By creating a better understanding of the 
research and development life cycle in ALS, it may be possible to understand 
the analogous CBS. A key aspect of the DDT&E life portion of the life cycle 
is that research activity never ends throughout the complete life cycle and 
that its products are always useful for strategic planning of future 
activities through a constant feedback loop. 

With this understanding, several cost related issues can be considered for 
collection by the Online Project Information System (OPIS) , to be described 
in more detail later. In particular, we need to collect the input 
information for whichever models we choose to utilize. With respect to ESM, 
we need mass, volume, power, cooling, crew time, and logistics information. 
Inputs vary with the models suggested by Jones, depending on the specific 
model used. They include the simplistic inputs, such as the mass of 
electronics and mechanical/structural mass, learning curves, launch dates, 
labor costs, perceived complexity, mission type, and whether or not the new 
system is a modification of a previous design (Jones, 2003; SAIC, 2004; NASA, 
2004c) . 

A significant portion of operations costs will be consumed by the provision 
of reliable and safe systems to the crew. Until highly developed 
technologies are available, it is challenging to make accurate predictions of 
the operations costs. It is a currently a fundamental research theme in 
reliability engineering to attempt to obtain reliability information on 
technology early in the life cycle of the technology 6 . With the appropriate 
reliability information for the components of the system, it will be possible 
to determine what the necessary redundant components may be and the necessary 
maintenance patterns should be. With this information operations costs can 
directly be determined. Analogously, reliability information early in DDT&E 


4 In some respects, perhaps these costs are not so difficult to determine, at 
least in aggregate form. The annual disbursement of funds to the ALS 
Program, NASA principal are of DDT&E for ALS missions, is a matter of public 
record, and can be determined with certainty. In aggregate form, it is most 
reasonable to spread cost data acquired in this manner equally over the 
entire ALS Program. LCC analysts should note, however, that such aggregate 
data would make it difficult to observe the costs and benefits of 
crosscutting versus highly focused research projects. Greater resolution may 
eventually be beneficial. This approach would provide costs incurred 
directly by NASA, which may or may not be the best perspective for the 
analysis. Such things as institutional taxes are wrapped into these figures 
and may or may not be interesting from some perspectives. 

5 Research, development, test, and evaluation should be conceptually identical 
to DDT&E to the reader. 

6 A short introduction to this research is provided in the appendix. 



phase of the life cycle can be tremendously useful, and should be stored for 
future analysis in a platform such as OPIS. 

Invariably during an analysis, especially one of such a grand nature as a 
LCC , some portion of the results may be viewed as suspect. This may be due 
to several reasons including inadequate input data, weak assumptions, lack of 
knowledge in a new area, or a rapidly changing environment. Sensitivity 
analysis considers the extent to which inputs to an analysis can vary without 
significantly affecting the result. This is often done by selecting the 
largest contributors to a particular result, as Drysdale did in his 2003 
paper. Then the analyst should identify the key input factors that lead to 
the result and vary them, observing the effect on the overall result. Key 
input factors can be identified by considering all those that may be expected 
to vary throughout the project life cycle or those that are determined based 
on assumptions or data of questionable accuracy. Depending on the structure 
of the models the various inputs will have differing effects on the results. 
For example, in ALS, varying the crew size can be expected to have a 
significant effect on the ESM of a system. Nearly every life support 
subsystem an ALS System is sized based on the number of crewmembers. It is 
reasonable to anticipate that the results of an ESM analysis should be 
especially sensitive to variations in crew size. To properly complete an 
analysis, LCC or otherwise, a sensitivity analysis should be performed to 
consider the range of potential outcomes that should be anticipated 
especially in areas where grand assumptions were utilized. 

To enable the analysis of sensitivity ranges anticipated data values are 
necessary, especially in the early stages of research and development. Such 
practices are already utilized in the Baseline Values and Assumptions 
Document, where nominal data values are presented with expected ranges 
(Hanford, 2002) . Other useful information that may be useful for sensitivity 
analysis describes how the data may vary. What probabilistic distribution 
best represents the data? What is the standard deviation or variance? 

Collection of Necessary Information for Decision Support 

A wide array of information describing the ongoing research activity in ALS 
is being compiled into the OPIS database (Levri, et al . , 2003). This 
information will be input and updated regularly directly by the principal 
investigators (PI) responsible for the research activity, creating a 
historical record of ALS activity. The baseline objective of the OPIS 
database is to make ALS research project information readily available to the 
members of the ALS Community. In doing so, several other possibilities 
emerge, including decision support for current researchers, prospective 
researchers, and the management of the ALS Program. It is proposed that the 
additional information described above could also be stored in OPIS to 
improve the analyses described previously. 

Decision support that would be beneficial for researchers is similar for both 
the current and prospective researcher. In each case, the researcher desires 
a more complete perspective of the state of the knowledge in ALS. With this 
knowledge, researchers can sculpt ongoing or proposed research to better 
fulfill the needs of ALS. In this respect, the enhancement of the transfer 
of information, similar to that currently provided on the World Wide Web, may 
be sufficient to enable decision support for researchers. Online tools and 
innovative graphical user interfaces can be made available to PI to expedite 
input of the information into the database. This is critical as the 
information to be requested by OPIS will be far-reaching and detailed in 



nature. In addition, often researchers from the ALS Community are asked to 
submit short biweekly and quarterly reports to provide regular updates of 
current activity, as well as yearly and final reports for wrapping-up of 
research findings over the long term. It has been proposed that the OPIS 
interface could be utilized to expedite the reporting process by 
interactively generating reports for the Pi's online. 

Once such data is stored in OPIS, it can be made searchable to the ALS 
Community. Dynamic interfaces can be constructed to attractively present the 
data to the users. It has been proposed to allow Pi's to prepare home pages 
to describe their projects. This will allow some latitude in personalization 
of how their work is presented to the community. Furthermore, OPIS can 
become a web portal to the ALS Community, where users not only could 
personalize how they are viewed by the community, but they can personalize 
how they view the community. An example of personalized web viewing is the 
MyYahoo! web page within the Yahoo.com® web portal (Yahoo! Inc., 2004) . Users 
can individually choose the content of the MyYahoo! web page to suit their 
personal web browsing habits. Therefore, users of OPIS could choose to view 
any portion of the OPIS database on a regular basis. For example, a 
wastewater-processing researcher may choose to view news regarding other 
relevant wastewater processing projects and provide links directly to 
relevant solid waste processing project home pages. In addition, based on 
user preferences and user browsing habits it is possible to generate 
individualized recommendations for areas within OPIS that users may interest 
the user (Linden, et al . , 2003). 

With respect to management, the provision of effective decision support will 
need to go beyond an effective web portal and generation of browsing 
recommendations, although these could certainly still be provided to 
management users. The most critical decisions made by management involve the 
distribution of resources, often in the form of research funding, throughout 
the ALS Community. Multiple metrics, including LCC, would be beneficial for 
this process. Not only will management require pertinent project information 
on a timely basis, but also information regarding how effectively the funds 
were used would be highly beneficial. 

Information that should be stored in OPIS should include the inputs to the 
various models used at the research project level. Cost data should include 
predicted costs made by researchers as well as actual costs incurred by NASA. 
Over time, with such historical data, ALS systems analysts can complete a 
greater portion of the CBS described above. With data describing a wider 
perspective of NASA ALS, the uncertainty of predictions can be determined, 
enabling sensitivity studies. 

With this information, it is proposed that several relationships could be 
generated using the data within OPIS, utilizing a combination of data mining 
techniques including fuzzy descriptors of quantitative values. The ideal 
relationships that would be useful for management decision-making include the 
models for CBS, however we are not limited to these. With data mining 
techniques, any aspect of the OPIS system could be investigated for useful 
information (Fayyad, et al . , 1996). Potential relationships are described by 
Eqns. (1), (2), (3), however the character of the actual relationships 

identified could take any conceivable form. 
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C * is cost to the ALS Program to improve the technology readiness level (TRL) 
of technology x to level y from y 0 , based on the states s if which are not 
necessarily directly related to technology x, and actions Similarly, T* 

is the time required to improve technology x to TRL level y from y Q , based on 
the same factors. AESM * represents the change in ESM when TRL is improved to 
level y from y 0 . 

If the relationships such as those described by Eqns . { 1 ) , ( 2 ) , and ( 3 ) 

could be ascertained, managers would have tremendous tools to utilize during 
decision making. For example, the expected change in the ALS Metric could be 
considered prior to investment in a technology, much like Drysdale proposes. 
Perhaps a relationship can be identified between PI cost estimates and actual 
expenditures. Relationships would not need to be deterministic in nature; in 
fact, if a range of values with confidence limits could be provided, managers 
would have much more information to deal with than they currently do. 

Consider further, the possibility that research centers or types of research 
centers, a state variable, could be identified as particularly proficient in 
a certain type of research or research dealing with technologies of specific 
TRL. Perhaps, the ALS Program would be most proficient at raising TRL from 
level 5 to 6, whereas universities are a better investment for TRL 1 s 1-4. 
Certainly, relationships describing time will be hugely beneficial when 
developing timelines and allocation of resources considering deadlines. 

Summary 

To maximize the potential for effective decision support within the ALS 
community, a wider array of metrics could be utilized. Development of a cost 
breakdown structure may go a long way towards providing the wider perspective 
of ALS necessary to enable effective decision support. The inputs to the 
various cost models within the cost breakdown structure describe the 
additional data that should be collected within the OPIS database. Data 
should be collected the project level and predicted as well as actual costs 
should be stored for later correlation. With such an expansive data set data 
mining may be a way to extract relationships that may otherwise be difficult 
to ascertain. In the short term, models such as those described by Drysdale 
and Jones may be utilized to ascertain the costs. In some cases, these 
models may not provide the resolution desired of a cost breakdown structure 
and, in some cases, the assumptions of these models are not completely 
transparent, which, will need to be considered in the near future. 

Some aspects of life cycle costs still need to be considered. In the past, 
dollar cost figures have not generally been made available to analysts, as 
they have been considered private and sensitive information. Furthermore, 
the cost of getting new ALS technologies approved for flight is extremely 
high, especially compared to a redesign or retrofitting of an existing flight 
approved technology. A life cycle perspective is exactly the perspective 
necessary to enable ALS technologies to prove their worth over existing NASA 
technology designed for short-term missions, however if cost data continues 
to be unavailable to analysts, either reasonably proxies will need to be 
identified or an arrangement needs to be created to share anonymous cost 
information with analysts. 



Another issue related to life cycle. costs has not yet been seriously 
considered, but may offer interesting perspectives for decision makers in ALS 
or even at the national level. Return on investment can be utilized to 
justify the potentially large investment for long-term missions to the Moon 
or Mars. Returns, however, are not always measured in dollar values. 

National pride, increased interests in space, science, mathematics, and 
technology are all significant returns, but it is challenging to convert 
these into returns on the initial investment. Nonetheless, NASA is very 
interested in returning value to the taxpayer in the form of technology 
transfer; it has been reported that returns on the Apollo program may have 
generated a return on investment for the US economy as high as 10 to 1. It 
should be expected that sustainability technologies for highly constrained 
scenarios, such as ALS missions, could enter the commercial marketplace. As 
natural resources on Earth dwindle the value of conservation should increase, 
making ALS technologies applicable even on Earth, perhaps creating a 
significant return on investment. Such an analysis would certainly be 
beneficial to justifying the expenditures on the ALS Program. 
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Appendix: Preliminary Reliability Assessment 

by: L.F. Rodriguez, D. Kortenkamp, A.B.O. Soboyejo, K.C, Ting, K. Pickering, 
F. Smith 

Two major obstacles remain for rigorous reliability assessment of ALS 
systems: (1) the lack of reliability data due to either prohibitive cost or 

low technology readiness and (2) the lack of a finalized system design 
perspective of an anticipated ALS system. The reliability analysis proposed 
here offers the opportunity to develop a preliminary reliability assessment 
of ALS systems and components and to develop techniques to consider the early 
phase reliability of a system. 

Current reliability analysis procedure does support the ability to develop 
reliability bounds on a system, even with limited information of the system 
configuration (Barlow and Proschan, 1974) . For example, such analysis can be 
'enhanced with the identification of cut sets and path sets, which can likely 
be identified. Cut sets are defined as a minimal set of components within a 



system that if each component within the set is failed, the overall system is 
ensured failure. Conversely, a path set defines a minimal set of components, 
which if operational, will ensure that the system will be operational, 
regardless of the condition of other components outside the pat set. It can 
be shown that k cuts sets K and s path sets S define the reliability 
bounds within which the true reliability of the system must lie. Eq. ( 4 ) 
depicts the current reliability bounds available through the use of cut sets 

n 

and path sets, where p i is known as the parallel of p x and is equal to 

1=1 
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With cut and path sets, the opportunity exists to further the existing state 
of reliability analysis. Can other analytically sound solutions be developed 
to further reduce the length of the reliability bounds currently obtainable? 
Perhaps, with some assumptions regarding the system configurations already 
mentioned above, further information might be garnered. In addition, it may 
be possible to develop more information by considering multiple states, such 
as off-nominal operational states, as proposed above with respect to the 
structure functions. 


For example, by coupling the limited data with anecdotal knowledge and 
perceptions of systems, it may be possible to utilize Bayesian techniques to 
leverage the sparse data. Bayesian decision analysis is often used in 
engineering analysis owing to its ability to incorporate human experience and 
knowledge with scarce experimental data in order to develop better-informed 
distributions regarding the actual distribution of the process under 
consideration (Soboyejo, Orisamolu, and Soboyejo, 2001; Soboyejo, et al . , 

1998; Soboyejo, 2002; Soboyejo, 2001; Sullivan, et al . , 2001). Eq. ( 5 ) 
demonstrates the formal statement of Bayes Theorem, where 

Pr(A)=£Pr(g,)Pr(A|B,). 
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In this methodology, prior beliefs are quantified into the Prior Distribution 
Pr («,) of the random variable. The experimental data is cast in the form of 
the Likelihood function, which determines the probability of the obtaining 
the set of experimental data under the assumption of the prior The 

expression results in the evaluation of the posterior (or updated) density 
Pr(g,U) , which can now be used for decision-making. 


With the availability of large amounts of data, the posterior density 
asymptotically resembles the density calculated by frequentist approaches 
(such as most likely estimation) . 

The choice of prior density and the experimental setup govern the ease of use 
of the Bayes Theorem. Several technologies (such as the Entropy method) are 



available for the derivation of the prior density for the purpose of this 
study. 


In addition, it would be useful to generate information regarding the failure 
distribution early in the investigation of a new technology. As mentioned 
above, collaboration has been established with existing wastewater processing 
and air revitalization research projects in the form of preliminary life 
testing of components and/or the consideration of existing data and the 
manner in which it was collected. For instance, is it possible to determine 
if a subsystem had an increasing or decreasing failure rate during its early 
experiments? Was its failure rate increasing or decreasing on average? This 
type of information is useful for reducing the length of reliability bounds. 


rt* Vi c 


:, this may be useful for developing system procedures*, such as 


maintenance policies, or perhaps for determining the optimal types of system 
redundancy . 
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